The Use of Instrumentation in Grammar Engineering

This paper explores the usefulness of a technique from software engineering, code instrumentation, for the development of large-scale natural language grammars. Information about the usage of grammar rules in test and corpus sentences is used to improve grammar and testsuite, as well as adapting a grammar to a specific genre. Results show that less than half of a large-coverage grammar for German is actually tested by two large testsuites, and that 10–30% of testing time is redundant. This methodology applied can be seen as a re-use of grammar writing knowledge for testsuite compilation. The construction of genre-specific grammars results in performance gains of a factor of four. 1